System for authoring and editing personalized message campaigns

ABSTRACT

A method and system for authoring templates for personalized video message campaigns in general, and personalized TV commercial campaigns in particular. The invention allows for campaign templates to be authored and edited in such a way that commercials can be automatically generated later on in the process, when the target audience and context are known, while at the same time adhering to the existing (post) production workflows and tools. The invention maintains assembly information for messages in a format that allows easy assembly of variations of commercials at any point in the creation, editing, post-production and delivery process.

RELATED APPLICATION

[0001] The present application claims the benefit of U.S. ProvisionalApplication No. 60/408,593 filed on Sep. 6, 2002, which is incorporatedherein by reference.

FIELD OF THE INVENTION

[0002] This invention is directed towards authoring, editing, andproduction systems for media and more particularly towards tools forcreating personalized TV commercial campaigns.

BACKGROUND

[0003] The creation of media campaigns such as television commercialshas become more complex as technology allows greater control andvariation of creation and delivery of such campaigns. In the case ofvideo messages, there are specific challenges with building templatesfor personalized message campaigns. Today's processes and products forvideo message production are focused on the creation of single, linearvideo messages, and do not easily allow for the creation of sets ofinter-related personalized video messages. May things are difficult,including the ability to build templates for personalized messagecampaigns, in such a way that it is easy to replace certain mediaelements based on user information, whilst maintaining the narrative ofall individual messages. Another difficulty is the ability to build userprofile templates, and link the user profile data to the variousdifferent media elements in a personalized message campaign template.Another difficulty is the ability to define environmental information,and link that to certain media elements in a personalized messagecampaign template.

[0004] As an example, the present-day process for creating (single) TVcommercials is illustrated in FIG. 1. A typical TV commercial startsfrom a campaign brief 20, which is typically created at the ad agency.The brief is a document that describes goal, target segment, and outlineof the commercial. The brief can be hand-written, or created with thehelp of a productivity tool like a word processor or other.

[0005] From there, the creative team designs a linear storyboard 22,which after being approved by the advertiser, becomes the shooting boardand script. The storyboard can contain sketches of the final media, orelectronic representations (thumbnails). The storyboard can behand-written, or created with the help of a productivity tool like aword processor or visualization program or other.

[0006] The shooting board is used by the production team during theshoot (Acquisition 24). Changes to the script and/or storyboard arechanged on location after approval by the agency. The shooting board canbe hand written, or created and maintained through a productivity toollike a word processor or visualization program or other. Sometimes(portable versions of) professional editing programs are used onlocation, so as to be able to create “rough edits” of the commercial (toensure that the material is good). An example would be the FinalCutPro(Editing Program by Apple Inc.) running on a portable Mac computer.

[0007] The resulting media elements are then finally edited 26 andlocked down. Audio elements are then recorded and graphics are created.Several professional audio and video (editing) tools are typically usedduring this phase. Examples are: MediaComposer (Editing Program byAvid), ProTools (Audio Editing Program by DigiDesign), etc.

[0008] All media elements are combined into a final linear story duringthe finishing stage 28. Several professional audio and video (finishing)products are typically used during this phase. An example is Symphony(Finishing Program, Avid). The last two phases (Editing 26 andFinishing) 28 are together referred to as Post-Production. The resultinglinear commercial is then either stored on a file server or mastered tovideotape.

[0009] Most present-day TV commercials are created this way, yet thereare numerous possible variations on this basic workflow, such as:

[0010] Multiple storyboards (or even complete commercials) are createdin parallel, the advertiser will then finally select one of them forfinal airing.

[0011] Iteration in the process, in other words certain steps arerepeated one or more times, until the result is approved by theadvertiser.

[0012] The storyboarding process actually includes the production of abasic low-cost version of the commercial (or at least an animation) soas to make the approval process easier.

[0013] Whatever variant on this process is followed, and whateverelectronic tools are used to support this, the process is fundamentallydesigned to produce linear stories where the narrative starts with aparticular shot, then moves to the next single shot, and so on until thestory is told. Further, although current video editing and finishingapplications allow for multiple video, audio and graphics tracks to beactive during the creation process, at the time of finishing, all ofthese tracks are resolved down to a final linear set, as shown in FIG.2. There is usually one video track 30, with graphics superimposed, andtwo or four audio tracks 32.

[0014] When this process is used to create different versions of acommercial, the finishing process actually results in a set of separatelinear TV commercials that are not related to each other anymore(although 80% of the content of the commercial may be identical). If itbecomes necessary to make changes in the “common” part of the TVcommercials, it is necessary to go back to the production process andre-do all different variants of the commercial. More specifically, theprocess does not allow for elements of the commercial to beautomatically changed/customized later on in the process (the commercialis already finished, and cannot be changed anymore without going back tothe production and finishing process). This makes it impossible toautomatically generate personalized versions of the commercial that arespecific for a given audience or context at (or close to) the time ofplay out.

[0015] Simply put, one of the limitations of the present day process isthat after post-production, all relations between media (differentversions) and data (demographics) are lost. All that is left is a (setof) linear TV commercial(s). This makes it impossible to automaticallychange the commercial and make it more relevant to a target audiencelater on in the process.

SUMMARY

[0016] The present invention includes a method for authoringpersonalized video messages more easily. An illustrative embodiment ofthe present invention is described in terms of a set of tools. Thepresent invention includes a method for creating a message campaign, themessage campaign allowing the creation of a plurality of differentmessages to targeted audiences. A specific targeted audience receives aselected one of the plurality of different messages based upon criteriaof the specific targeted audience. The method includes providing aplurality of media segments, the media segments for assembly into theplurality of different messages to targeted audiences, wherein at leastone of the media segments is interchangeable with another one of themedia segments. The media segments may be created, preexisting orcreated at a later time. The method also includes providing assemblyinformation regarding how the plurality of media segments may beassembled to create the different messages. The assembly information isthen associated with the plurality of media segments. This assemblyinformation and media segments may then be packaged up to be broadcastor dispersed.

[0017] Further, a message for a specific targeted audience may beassembled at a later time, the message being assembled based upon theassembly information, the plurality of media segments, and oninformation regarding the target audience. An example is selecting mediasegments based on a target audience's age or preferences, and using thatmedia segment along with other related or unrelated media segments toassemble the message. Further, one or more media segments used toassemble the messages may be created at the later time. An example istitling for a message or commercial may be created and assembled withthe message right before it is shown to a target audience.

[0018] The assembly information may include rules for use at the latertime, the rules for use in determining which of the plurality of themedia segments to use in assembling a message for the specific targetedaudience, based on the information regarding the target audience. Therules can include “default conditions” for determining which of theplurality of the media segments to use when no appropriate informationregarding the target audience is available. A subset of the plurality ofmedia segments can form a default generic message.

[0019] Media segments can include audio, video, voice overs, andbackground music. Each type of segment can be assembled with othersegments based on information or conditions imposed by the assemblyinformation. The assembly information can include data representing timesegments; the media segments, and conditions. This assembly informationforms a network which maintains a matrix-like overview of the mediasegments, their timing, and how they may be combined with other mediasegments in order to assemble the messages. In one illustrativeembodiment, the message for a specific targeted audience is assembled ina set top box for a television receiver contemporaneously withdisplaying the message to the specific targeted audience.

[0020] The specification includes illustrative embodiments focusing on atelevision commercial as a specific form of personalized video message,yet it is easy to see that the approach is equally valid forpersonalized video messages in general (e.g. personalized news casts)and even personalized messages in general (e.g. personalized audiomessage, personalized multi-media presentation).

[0021] An illustrative embodiment defines the concept of a matrix torepresent sets of video messages in general, and a personalizedcommercial campaign template in particular. The matrix defines TVmessages in terms of defaults and options for video, audio, narration,titling overlay, and other media elements. The matrix associatessegmentation data with the various media options. Once the matrix iscreated, produced, and finished, it can be played out at differentdelivery networks, to different target audiences, in different contexts,in which case it will each time generate a specific TV commercial thatis customized to audience and context. A possible implementation of amethod for authoring personalized messages using the matrix conceptspecifically focuses on a data model (network) that represents thematrix-based message template. The data model typically is objectoriented, and can be stored in relational database or other to form adata repository that is shared among tools. The tools will use the datain the repository, and interface with the workflows and tools that arecurrently being used during production of TV messages.

[0022] One advantage of the illustrative embodiment over existing (post)production tools and products, is that it allows for campaign templatesto be authored in such a way that the commercial can be automaticallygenerated later on in the process, when the target audience and contextare known, while at the same time adhering to the existing (post)production workflows and tools.

[0023] Another advantage is that it is possible to produce and finishsets of (related) TV commercials more efficiently. For instance, if aset of related TV commercials (in a matrix-based system) share a pieceof media (e.g. the opening scene), and that piece of media needs to bere-worked (e.g. re-edited), then the tools and system as described inthis document will allow for this to be done by one single edit session,after which the update will effectively “ripple” its way through thematrix of related commercials (whereas in the current process, one wouldhave to individually re-work all individual messages). This brings anefficiency advantage to (post) production of related TV commercials.

[0024] Yet another advantage over the current process for producing TVcommercials lies in the moment of media assembly. In the currentprocess, TV commercials are singular after production, in other words:the final choice of media is made during the production and finishingprocess. In fact the post-production can only be finalized if all mediachoices are made. The concept of a narration matrix according to oneembodiment of the present invention allows sets of related commercialsto be post-produced, effectively delaying the moment of final mediachoice and assembly until after the post-production process (to a pointwhere the audience and context are better known).

BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The foregoing and other features and advantages of the presentinvention will be more fully understood from the following detaileddescription of illustrative embodiments, taken in conjunction with theaccompanying drawings in which:

[0026]FIG. 1. shows the process of workflow for commercials in the priorart;

[0027]FIG. 2 displays the linear narrative result of the workflowprocess of FIG. 1;

[0028]FIG. 3 shows a video matrix according to the present invention;

[0029]FIG. 4 shows an example video matrix with multiple shots peroption;

[0030]FIG. 5 shows an example nested video-matrix;

[0031]FIG. 6 shows an example double narrative nested matrix;

[0032]FIG. 7 shows an example matrix with restrictions;

[0033]FIG. 8 shows an example of fully disjunct commercials in a matrix;

[0034]FIG. 9 shows an overall matrix according to the present invention;

[0035]FIG. 10 shows a titling matrix according to the present invention;

[0036]FIG. 11 show an example titling matrix with parameters;

[0037]FIG. 12 provides an overview of a suite of authoring toolsaccording to an illustrative embodiment of the present invention;

[0038]FIG. 13 provides a mapping of authoring tools on workflow aspreviously described in FIG. 1;

[0039]FIG. 14 shows samples of simple networks according to the presentinvention;

[0040]FIG. 15 shows a screen shot of a storyboard tool according to anillustrative embodiment; and

[0041]FIG. 16 shows a postman tool screen shot according to anillustrative embodiment.

DETAILED DESCRIPTION

[0042] U.S. patent application Ser. No. 09/545,524 entitled TEMPLATECREATION AND EDITING FOR A MESSAGE CAMPAIGN and assigned to VisibleWorld Inc., describes a novel system for creating personalized messagesfor audiences and is incorporated herein by reference.

[0043] The present invention introduces a new, efficient way ofauthoring personalized TV commercial campaign templates, based on theconcept of Matrix-Based Narrative. Matrix-Based Narrative provides theability for commercial campaign templates to be authored in twodimensions (time and choice), and the resulting finished commercialcampaign template will still include these two dimensions (as opposed tothe current approach, where a finished commercial only has the timedimension, since there is no element of choice anymore). Morespecifically a Matrix-Based Narrative contains a linear (default)commercial, but for one or more elements (called “slots”) of thiscommercial it contains multiple play-out candidates (called “options”).

[0044] A simple example of such a matrix according to the presentinvention is shown in FIG. 3. For the sake of simplicity, this exampleonly considers the video-part of the commercial (ignoring audio- andother elements for now). The example illustrates a simple commercial,consisting of five shots, where shot 2, 4, and 5 are actually slots withmultiple options. It is easy to see that the video-matrix defines adefault commercial 32 (composed of Shot 1, Shot 2A, Shot 3, Shot 4A,Shot 5A) as well as twenty-three alternative commercials (generated bysequencing the various permutations of options for each slot). Totalnumber of commercials contained within the video-matrix is twenty-four(=1×4×1×2×3).

[0045] In order to understand how this video-matrix can be applied todefine any set of commercials, it is helpful to look at some moreexamples. First, consider an example shown in FIG. 4, that is similar tothe previous example, yet allows for a slot to contain an arbitrarynumber of shots, as long as the length of the various options for agiven slot is identical. In this example there is a default commercial34 (Shot 1-1, Shot 1-2, Shot 2A, Shot 3, Shot 4A, Shot 5A1, Shot 5A2) aswell as 23 alternative commercials.

[0046] Another example as shown in FIG. 5 is where there is nesting ofslots (in other words: a choice within another choice). In this example,the default commercial 34 consists of the sequence (Shot 1, Shot 2A,Shot 3). The alternative commercial has Shot 2A replaced, resulting inthe sequence (Shot 1, Shot 2B/1, Shot 2B/2 a, Shot 2B/3, Shot 3).However, since the second shot in the 2B-sequence is again a slot withmore options, there are two more valid commercials in this video-matrix,namely (Shot 1, Shot 2B/1, Shot 2B/2 b, Shot 2B/3, Shot 3) and (Shot 1,Shot 2B/2 c, Shot 2B/3, Shot 3). So the total number of commercials inthis video-matrix is four (=1×(1+(1×3×1))×1).

[0047] A slightly more complex example of nested slots is illustrated inFIG. 6. In this example the default commercial 34 is (Shot 1, Shot 2A/1,Shot 2A/2 a, Shot 2A/3, Shot 3). There are two slots within thatmessage: Shots 2A together represent a slot, yet within that also Shot2A/2 a represents a (nested) slot. The same is true for shots 2B andshot 2B/2 a. Therefore, in addition to the default commercial 34, thefollowing sequences represent valid commercials in the video-matrix:(Shot 1, Shot 2A/1, Shot 2A/2 b Shot 2A/3, Shot 3), (Shot 1, Shot 2B/1,Shot 2B/2 a, Shot 2B/3, Shot 3), (Shot 1, Shot 2B/1, Shot 2B/2 b, Shot2B/3, Shot 3), (Shot 1, Shot 2B/1, Shot 2B/2 c, Shot 2B/3, Shot 3).Total number of commercials in this video-matrix is five(=1×((1×2×1)+(1×3×1))×1).

[0048] A video-matrix may contain many sequences (permutations of shotsand options), some of which may be invalid. In the present invention itis possible in a video-matrix, to specify which permutations areinvalid. This effectively reduces the number of commercials within avideo-matrix, as shown in the example presented in FIG. 7. Thisvideo-matrix would normally contain six (1×2×3×1) different commercials.But for this example, assume that for some reason the author of thevideo-matrix does not want the combination of Shot 2A with Shot 3B tooccur in any commercial. The author would then put a limitation in thevideo-matrix that renders invalid any commercial that contains thiscombination of shots. The result would be that the video-matrix now“only” contains five (=(1×3×2×1)−1) valid commercials, namely (Shot 1,Shot 2A, Shot 3A, Shot 4), (Shot 1, Shot 2B, Shot 3A, Shot 4), (Shot 1,Shot 2B, Shot 4), (Shot 1, Shot 2C, Shot 3A, Shot 4), (Shot 1, Shot 2C,Shot 3B, Shot 4).

[0049] By combining all these techniques and examples, and applying themto real sets of commercials, it is clear that any set of commercialsthat have common elements can be expressed as a video-matrix. Moreover,even in the “worst” case of a set of commercials that have no commonelements at all, it is possible to combine the commercials in avideo-matrix. One would simply create a “slot” that covers the fulllength of the commercial, and the various commercials would representfull and complete options within that slot, as is illustrated in FIG. 8.This video-matrix contains exactly 2 valid commercials, namely thedefault (Shot 1, Shot 2, Shot 3, Shot 4, Shot 5) and the alternative one(Shot 6, Shot 7, Shot 8, Shot 9).

[0050] The disclosure so far has only described the video part of TVcommercials. The present invention is not limited to video, but alsoextends to audio and other multi-media elements. For this purpose,additional dedicated matrices for titling, voiceovers, and music areintroduced. Conceptually these matrices all describe different “planes”or “views” of the template, where the resulting overall commercials arebuilt by combining all these “planes” into discrete commercials. Thefour matrices can all have slots and options independent of each other.An example of an overall-matrix 36 is shown in FIG. 9. Thetitling-matrix 38 contains the various graphics and/or titles that aregoing to be part of the commercial. The voiceover-matrix 40 contains thevoiceover clips, and the music-matrix 42 contains the background musicclips. The four matrices are combined into an overall-matrix 36 thatrepresents the final personalized commercial campaign template includingvideo, audio, and graphics.

[0051] This overall matrix describes a set of TV commercials withtwenty-four (=1×4×1×2×3) video messages, two titling messages, four(=1×4×1) voiceover messages, and two music messages. Theoretically thetotal number of different TV commercials contained within theoverall-matrix is therefore three hundred and eighty-four (=24×2×4×2).However in reality there will typically be restrictions built into thematrix, for instance certain voiceover options can only be combined withcertain video options, so that the real number of valid TV commercialsin the matrix would typically be less than three hundred andeighty-four.

[0052] Note that more matrices may be added for specific multi-mediaelements if required. For example there could be a matrix forinteractive application data that is part of a commercial, further therecould be multiple video matrices for devices that combine multiple videosources, etc.

[0053] An example of a titling matrix is shown in FIG. 10. As discussedin the previous section, titles and graphics can be expressed as mediaelements in a matrix, so that there can be variation in the titling andgraphics of a matrix of TV commercials. This simple matrix represents asituation with no titling-overlay 44 in the beginning of the commercial,followed by a default title overlay “Title 1”, with an alternative titleoverlay “Title 2”. The end of the commercial, again carries notitling-overlay 44.

[0054] In addition to this, the titling matrix 38 enables one toparameterize the title overlay by including references to values thatwill be computed later on (from database tables, or other sources ofinformation). Consider the example in FIG. 11. This matrix represents asituation with no titling overlay 44 in the beginning of the commercial,followed by a default title overlay 46 “Visit your store today at %address %”, where % address % is referring to a store address that iscomputed later on (from a database or otherwise). The alternative titleoverlay 48 is “Order Today! Call % phone_nr %”, where % phone_nr %refers to a phone number that is computed later on. If the computationof store addresses, resp. phone numbers results in NR_ADDRESSESdifferent store addresses, and NR_PHONE_NUMBERS different phone numbers,then it is clear that this titling matrix representsNR_ADDRESSES+NR_PHONE_NUMBERS different titling overlay sequences forthe associated commercials.

[0055] The concept of a matrix, as defined in the previous sections,basically introduces a representation for personalized TV commercialcampaign templates. For example, a matrix can represent a campaign,where the actual TV commercial that is shown to an individual (or group)is customized to, and specific for that individual (or group).

[0056] In order to facilitate this, the matrix concept is extended withthe concept of a target. A target is an individual, or a group ofindividuals, that may view one of the TV messages in the matrix. The setof all targets represents the set of all potential viewers of TVcommercials from the matrix. A target is defined in terms ofsegmentation parameters. Segmentation parameters may be demographicparameters (e.g. income level), environmental parameters (e.g. currenttemperature outside), contextual parameters (e.g. which TV channel iscurrently being watched), or any other parameters. Each segmentationparameter may have a set of values. The matrix allowssegmentation-parameter values to be attached to slot-options. Theseassociations can be simple (e.g. “slot option B1” is associated with“income=high”) or may be combined in more complex expressions (e.g.“slot option B2” is associated with “income=low ANDcurrent_temperature>50”). Since any target can be represented as a setof values for the segmentation-parameters, this mechanism can be used toassociate one or more targets with each TV commercial in the matrix.

[0057] It is important to note that this mechanism may work with any setof segmentation parameters (with finite, or even infinite values), andwith any expression syntax for combining values (Boolean logic, fuzzylogic, rule-based systems, etc.).

[0058] Personalized TV commercials may be delivered over variousdistribution networks. Examples of distribution networks include: analogcable networks, digital cable networks with set top boxes with orwithout hard disks, digital satellite networks with set top boxes,internet streaming, Video-On-Demand-systems, distribution on hard mediasuch as CD or DVD discs, etc.

[0059] Certain networks may impose certain restrictions on personalizedTV messages. These restrictions can be fundamental (e.g. a certainnetwork does not support a certain type of audio formats) or practical(e.g. a certain network operator wants to allocate only finite resourcesto a personalized TV campaign in terms of storage and/or bandwidth).This can easily be represented in the matrix as a set ofdelivery-related restrictions. These restrictions can be bandwidthrelated (e.g. the total number of different options for slots at aspecific point in time not to exceed X), storage related (e.g. the totalsize of all media fragments in a matrix not to exceed Y), or other.

[0060] It is also important to point out that, since a matrix may bedesigned for different delivery networks, with differentcharacteristics, a matrix may have different sets of restrictionsassociated with it, one per delivery network.

[0061] Tools used in the present day for media creation and editing willbenefit from the present invention. Presently, linear commercials areauthored and finished based on the workflow as depicted in FIG. 1. Thisworkflow is supported by a variety of tools, such as Non-Linear-Editors(NLEs), Audio Editors, etc. Matrix-based templates according to thepresent invention provide enhancements to this workflow and the toolsthat support it. The present day tools, by themselves, are not properlyequipped to build and finish matrix-based templates. Descriptions oftools and illustrative embodiments that support matrix-based templatebuilding will now be disclosed. The tools are designed in such a waythat they augment the present day production processes and tools(instead of replacing them). This makes their initial acceptance mucheasier. Alternative implementation strategies are discussed furtherbelow.

[0062] An illustrative embodiment of a high-level design of a tool setis illustrated in FIG. 12. The basic premise is to introduce a suite oftools that assist in the workflow in order to facilitate the authoringof matrix-based templates. The tools interface with the processes andproducts that are currently used in production and post-production. Eachtool is specifically designed to assist in one (or two) phases of theworkflow (as illustrated in FIG. 13). They share information through acommon repository 50.

[0063] The fundamental goal of the tools is to build and maintain a datamodel that defines the relation between media, data, and differentvariants of the TV commercial (which is something the currently existingtools cannot do). This is done in an illustrative embodiment bymaintaining a data model (called network or template) that links allmedia and all data together. This network is defined in early stages inthe process, and is refined and finished in the final stages. A networkis shared between all tools. A network consists of three basic buildingblocks (called elements or objects) that can be combined to representany matrix-template:

[0064] Media Elements represent one or more pieces of media. This can bea video shot, a series of video shots, audio, titles, pictures, etc. Amedia element may also contains meta-data on the media elements itrepresents (for instance what actors appear in it, descriptive text,copyright information, etc.).

[0065] Conditions represent expressions that allow for certain parts ofthe network to be conditional upon audience data (such as demographics).The expressions can be logical expressions, fuzzy logic, or any otherconditional expression language. The data can be demographics data,real-time data, or any other type of data that is either stored indata-bases, or is made available in another way. Conditions also havethe concept of priority. So two conditions may both be true, but theymay have different priority.

[0066] Time Nodes represent specific times in the network. Time Nodescan have relative time or absolute time. They typically represent thetiming of a choice in the network.

[0067] A number of examples of networks are shown in FIG. 14 (roundedsquares marked M represent media elements, circles marked T representtime nodes, and diamonds marked C represent conditions).

[0068] The first example (A) starts with a simple network that startswith a time node (T1), followed by a media element (M1), and ends withtime node (T2).

[0069] The second example (B) shows a similar network, but now the mediaelement is associated with condition (C1). If C1 is true, the effect ofexample B is identical to example A. If C1 is false the network is emptyOust has a begin time and end time). In the general case media elements(or sequences of media elements) have conditional associated with them,but in case that condition is always true, it may be left out. In otherwords, example A could be represented as example B with C1=always true).

[0070] The third example shows a start time node (T1), followed by mediaelement (M1), followed by time node (T2), followed by media element(M2), ending with time node (T3). This represents a simple timedsequence of two media elements.

[0071] The final example (D) starts with time node (T1), followed bymedia element (M1), followed by time node (T2). After this, there aretwo possible media elements, M2 and M3, both followed by end time node(T3). Media element M2 is associated with condition (C1), and mediaelement (M3) is associated with condition (C2). This network representsa timed conditional sequence of media elements as follows: TABLE 1Evaluation of Conditions Resulting Sequence C1 = false AND C2 = false M1C1 = true AND C2 = false M1, followed by M2 C1 = false AND C2 = true M1,followed by M3 C1 = true AND C2 = true AND M1, followed by M2priority(C1) > priority(C2) C1 = true AND C2 = true AND M1, followed byM3 priority(C2) > priority(C1)

[0072] These examples illustrate the basic concepts of a network, andsuggest how a network may be used to express more complex matrix-basedtemplates.

[0073] A more formal definition of a network is as follows. Networks arebuilt from the following 3 basic elements (which can be thought of asobjects or object classes):

[0074] Media Elements represent media sequence(s). They basicallycontain references to the media in some form, as well as meta data onthat media. There are different types of media elements, one for videomedia, titling media, audio media, narration media (more can be added ifneeded).

[0075] Conditions represent expressions that may refer to data indatabases, or any other data.

[0076] These expressions will render true or false, depending on thevalues of the underlying data, as well as a priority. The priority isused to select between multiple conditions that render true at the samepoint. The expressions may be in any expression language.

[0077] Time Nodes represent discrete points in the timeline of acommercial. There is a start time node, an end time node, and there canbe any number of time nodes in between.

[0078] Elements in a network may be connected as follows:

[0079] A network always starts with at least one start time node.

[0080] A time node can be followed by another time node. This representsa situation where a certain time interval in the commercial is (still)empty.

[0081] A time node can be followed by a media element. This represents asituation where the media elements start at a specific time in thecommercial.

[0082] A time node can be followed one or more conditions. Thisrepresents a situation where there is an element of choice at a certainspecific time in the commercial. When executed, the condition thatrenders true and has the highest priority, prevails.

[0083] A time node can be followed by a media element as well as one ormore conditions. This represents a situation where there is an elementof choice between the default (media element) and other options(conditions). When executed, the condition that renders true and has thehighest priority prevails. In case all conditions render false, thedefault prevails.

[0084] A condition is always followed by a media element. Thisrepresents the fact that the conditional expression is associated withthat specific media sequence.

[0085] A media element is always followed by one time node. Thisrepresents the fact that a media sequence always ends a specific time.

[0086] Multiple media elements may be followed by the same time node.This represents an end of a slot.

[0087] A network always ends with at least one end time node.

[0088] A network can be stored in different ways. Since a networkbasically consists of a collection of elements and relations betweenthese elements, it can be stored in a relational database, in aobject-oriented database, in XML files, in proprietary formats, etc. Themain thing is that there is a place to store networks, and to share thembetween the various tools that access and manipulate them. In theremainder of this chapter we will refer to this place as the SharedRepository.

[0089] In the illustrative embodiment, a network is a representation ofa personalized commercial campaign template. It is built, manipulated,and finalized through a set of tools. Each tool has specific tasks inbuilding or refining the network. Each tool will also enforce specificconstraints and constructs, so that at the end of the process, theresulting network will represent a valid campaign template.

[0090] The StoryBoard Tool 52 essentially builds the network (createmedia elements, time nodes, and conditions, and connect them in ameaningful way). The resulting network will be complete, although themedia elements may not yet have media associated with them (or temporarymedia), the conditions are in place (although the final underlyingdatabase functions may not be in place yet), and the time nodes are inplace (although they may only indicate rough timing).

[0091] The User Interface of the Tool represents the network structurein an intuitive and user-friendly way. An implementation of the UserInterface may combine three different windows: The main network windowprovides a representation of the network. The network is divided in fourdifferent sub-networks (video, audio, titling, narration). The variousmedia elements can be associated with (and shown as) scanned images,stills, or any other representations. The network can be showncompletely, or sub-networks may be hidden (e.g. only look at video andnarration, hide titling and audio). There is an option to select andbrowse single paths through the network. Time nodes are not shownexplicitly, but are implicitly derived from timing of various mediaelements. Although this time is not going to be final (final timing istypically determined during post-production), they may be used as roughtime indicators. The matrix can be shown in non-timed mode (e.g. eachshot has same length) or in absolute time-mode (length of shots shownproportionally).

[0092] The data/selection window is essentially a representation ofexpressions and data within expressions. The window contains arepresentation of the various segmentation parameters that are availableto the campaign designer (for instance “income=high” or “age=young”).These parameters represent condition objects. They can also be combined(for instance “income=high AND age=young”) in which case a new conditionobject is created to represent the combined expression). The variousparameters and expressions can be dragged and dropped to media elementsin the main network window. This has the effect of creation a newcondition and inserting it in the network before the media element.

[0093] The media window can be used to browse available media elements.These elements can be scanned images, still images, etc. They basicallyrepresent the media elements that are available for building a network.Any element can be dragged and dropped onto the main network window.This will have the effect of creating a new media element, inserting itat a certain place in the network (also creating one or two time nodeswhen needed).

[0094] A screen shot of a storyboard tool according to an illustrativeembodiment is shown in FIG. 15. A main network window is shown on thefight side, and a set of stacked media windows is on the left side. Adata/selection window is not shown in this example. The StoryBoard maybe used to create a brand new network, yet can also be used to refine ormodify an existing network. In reality it will probably be used inseveral sessions to go from initial concept to final network. TheStoryBoard Tool 52 will ensure that, at all times ensure that thenetwork is structurally correct (e.g. following the formal networkspecification rules as previously described).

[0095] In addition to this, the StoryBoard Tool 52 can also checkagainst delivery constraints. Delivery constraints are specificlimitations related to bandwidth or storage or formats that are imposedby specific delivery networks. Examples are: certain digital cablenetwork may not support certain audio types, or a certain satellitenetwork may have bandwidth limitations that restricts the number ofsimultaneous slot options, etc. The StoryBoard Tool 52 will check theseconstraints by inspecting the network data structure, and issuingappropriate warnings where necessary.

[0096] In an illustrative embodiment, a StoryBoard Tool 52 may also havethe following features:

[0097] Printing features, including the ability to print high-quality,customized reports of the campaign template.

[0098] Animation features, including the ability to select a target (orset of segmentation parameter values) and play out an animated versionof the associated commercial.

[0099] The ability to “pre-package” certain parts of the network (interms of structure, or pre-packaged segmentation parameters, or anycombination thereof), and be able to save these as templates (which canbe used as starting points for building several other networks).

[0100] The ability to (automatically or manually) check the matrixagainst delivery constraints (including specific warnings about wheredelivery constraints are violated and how that could be fixed).

[0101] The ability to import segmentation data from other systems andtools in general, and from data analysis and data segmentation toolsspecifically.

[0102] An illustrative embodiment of a Location Tool 54 assists producerand production crew in acquiring material for a matrix-based commercial.The tool needs a valid network structure as input (typically built bythe StoryBoard Tool 52). The tool will analyze the network, and extractall media elements from it, and present an “electronic checklist” of themedia that needs to be filmed, recorded, computer generated, orotherwise created. This checklist is interactive, so that the productioncrew can “electronically check off” the media that are acquired. As faras the network structure, the Location Tool 54 may only (optionally)update the existing media elements to refer to the newly acquired rawmedia. Typically the tool does NOT change the network in any other way.

[0103] An illustrative embodiment of a Titling Tool 58 assists creativestaff and editors to design the titles that are used in the campaign.While the Storyboard Tool 52 and Postman Tool 56 define when titlesappear in the messages, the Titling Tool 58 focuses on the actual screenlayout, effects, precise timing, etc. It allows the appearance of thetitle to be designed, while the content of the title can later on becustomized/changed according to the play-out rules and data.

[0104] The Titling Tool 58 typically needs a valid network structure asinput (typically built by the StoryBoard Tool 52). The tool will onlyaccess the titling sub-network. It analyzes the titling sub-network, andallow creative designers (on a per titling media element basis) todesign the exact screen-layout of the element, precise timing, addeffects, etc. As a result, the titling sub-network will be updated asfollows:

[0105] Time nodes within the titling sub-network may be updated (toreflect changes in precise timing of the individual titles).

[0106] The titling media elements will be updated to refer to the newlydesigned titling formats, as well as additional meta data on effectsetc.

[0107] Typically the Titling Tool 58 does not change the network in anyother way.

[0108] An illustrative embodiment of a PostMan Tool 56 is used to do thefinal updates of the network structure and make it ready for furtherautomatic processing and delivery. The Postman Tool 56 plays a centralrole during post-production (editing and finishing). It interfaces withall other products that are used in production and post-production. Theuser interface of the tool represents the network structure in anintuitive and user-friendly way. An implementation of the User Interfacemay combine three different windows:

[0109] The main network window as a representation of the network. Itlooks very similar to the main network window in the StoryBoard Tool.The network is divided in 4 different sub-networks (video, audio,titling, narration). The various media elements can be associated with(and shown as) scanned images, stills, or any other representations. Thenetwork can be shown completely, or sub-networks may be hidden (e.g.only look at video and narration, hide titling and audio). There is anoption to select and browse single paths through the network. Time nodesare not shown explicitly, but are implicitly derived from timing ofvarious media elements. The matrix can be shown in relative time-mode(e.g. each shot has same length) or in absolute time-mode (length ofshots shown proportionally). All media elements in the main networkwindow have a status label. The status label indicates whether or notthe specific media sequence is finalized and valid. So for instance ifthe media sequence for a certain media element is still un-edited itwill be marked with a red label, if the media sequence is edited andfinished and final it will be marked with a green label. See also thePostMan Tool algorithm description below.

[0110] The selection window is essentially a representation of thevarious conditions and “paths” through the network. It is used to selectcertain parts of the network for production and post-production. Thelist will typically contain a “default” path, and all alternative paths.There is color-coding in the list to indicate paths that are finished,are still to be done, or are in conflict.

[0111] The media window can be used to browse available media elements.These elements can be scanned images, still images, etc. that can beassociated with the media elements in the network. These images are fordisplay only, and will not affect the final media.

[0112] An example screenshot of the PostMan Tool is shown in FIG. 16. Itshows the main network window at the top, the selection window at thebottom right, and a media window at the bottom left. The PostMan Toolworks from an existing, valid network structure, which it manipulates asfollows:

[0113] Time nodes within the titling sub-network may be updated (toreflect changes in precise timing of media elements).

[0114] The media elements will be updated to refer to the final media,as well as additional data on effects etc.

[0115] Typically the tool does not change the network in any other way.The tools will produce the final network structure that is ready forfurther processing and delivery.

[0116] In an illustrative embodiment, the basic algorithm of the PostManTool is:

[0117] 1. Mark all media elements in the network as invalid

[0118] 2. Select next (partly) invalid media sequence from the network

[0119] 3. Interact with post-production tools to produce a finished,post-produced version of that media sequence.

[0120] 4. Check the finished media sequence for consistency with thenetwork.

[0121] 5. If inconsistent: trigger user, and present with options forrepairing the inconsistency

[0122] 6. If consistent: update the network structure to reflect thenewly finished media sequence

[0123] 7. If there is at least one invalid media element in the network:go to step 2.

[0124] Explanation:

[0125]1. The initial step is to render all media elements as invalid(meaning: not finalized). The status of a media element is kept in thenetwork structure. The PostMan Tool will mark all media elements (in itsmain network window) wit a status label reflecting its status, which canbe either valid (green label) or invalid (red label).

[0126]2. During this step a next media sequence will be selected fromthe network. A media sequence is a complete set of media elements, andcan either be video, audio, narration, titling, or any combinationthereof. A sequence can consist of any combination of valid and invalidelements. The sequence can be hand selected by the operator of the tool,yet the tool will suggest the most optimal next sequence. The initialsuggestion will be the default commercial. After that the sequence withthe biggest number of invalid media elements will be suggested.

[0127]3. During this step a media sequence will be transferred to theappropriate post-production tool, where it will be produced a/orfinished. An example is a sequence of video, audio, narration, andtitles that will be transferred into a Non Linear Editor (NLE) forediting. The transfer can be through any interface that allows for theNon Linear Editor to be “loaded” with a media sequence to be edited. Itis important that all elements in the media sequence are labeled withunique identifiers (by the PostMan Tool), and that these identifiers arekept during the editing process, and return with the media sequence whenreturning from the NLE to the PostMan Tool. One possible implementationof the communication interface between PostMan Tool and post-productiontools is AAF (Advanced Authoring Format), which provides a standardprotocol for transferring media structures.

[0128] 4. Once the finished, post-produced media sequence is returnedfrom the post-production tool to the PostMan Tool, the latter will checkthe consistency between the sequence and the network structure (itoriginated from). This check is by done by taking the unique identifiersof the media elements, and using them to “find back” the original mediasequence in the network, and then comparing the two for consistency.

[0129] 5. The consistency check may reveal one or more of the followingproblems:

[0130] a. Media elements may have wrong identifiers, or they are out oforder. This error will be presented to the user, and unless the userre-arranges and/or modifies the identifiers, this is a non-correctableproblem.

[0131] b. Media Elements have no identifier. This is a correctableproblem, since the PostMan Tool will simply “add” these elements to theclosed possible element that does have a known identifier.

[0132] c. The timing of the new sequence is different from the network,and is incompatible with timing of already validated media elements(e.g. different options for the same slot now have different lengths).This will be flagged to the user, who will have to choose which timingis valid.

[0133]6. In case of a non-conflicting situation, the network structurewill be updated to include the newly finished media sequence. This mayinclude: changing time nodes, and updating media elements to refer tothe final media, and adding metadata.

[0134]7. Repeat these steps until all media elements in the network arevalid at the same point in time. In this case the network is ready forfurther processing and delivery.

[0135] The tools as described in the previous paragraphs are just onepossible implementation of the present invention for a system forauthoring (matrix-based) personalized messages. Alternativeimplementations are possible. There are at least two differentapproaches to alternative implementations:

[0136] 1. Leaving the current workflow and tools intact, yet augmentthem with a separate layer of authoring tools (Oust like theimplementation as previously described), yet with a differentimplementation of the layer of authoring tools. Different toolsimplementations include: combine multiple tools in one tool, or break-upthe tools functionality over more (or different) tools. It could alsomean that certain tool functions are more intimately “added” to theexisting post-production products (for instance through plug-ins in NonLinear Editing systems).

[0137] 2. Creating a completely new implementation, where currentpost-production workflows and tools are replaced with new workflows andtools that combine the original functions with the functions ofmatrix-based messages. An example of this is to create a new Non LinearEditing system that features editing matrix-based messages. These twoapproaches can also be mixed.

[0138] Although the invention has been shown and described with respectto illustrative embodiments thereof, various other changes, omissionsand additions in the form and detail thereof may be made therein withoutdeparting from the spirit and scope of the invention.

[0139] What is claimed is:

1. A method for creating a message campaign, said message campaign allowing the creation of a plurality of different messages to targeted audiences, wherein a specific targeted audience receives a selected one of said plurality of different messages based upon criteria of said specific targeted audience, said method comprising: providing a plurality of media segments, said media segments for assembly into said plurality of different messages to targeted audiences, wherein at least one of said media segments is interchangeable with another one of said media segments; providing assembly information regarding how said plurality of media segments may be assembled to create said plurality of different messages to targeted audiences; and associating said assembly information with said plurality of media segments.
 2. The method of claim 1 wherein a message for a specific targeted audience is assembled at a later time, said message being assembled based upon said assembly information, said plurality of media segments, and on information regarding said target audience.
 3. The method of claim 2 wherein at least one media segment used to assemble one of said specific messages is created at said later time.
 4. The method of claim 2, wherein said assembly information includes rules for use at said later time, said rules for use in determining which of said plurality of said media segments to use in assembling a message for said specific targeted audience, based on said information regarding said target audience.
 5. The method of claim 4 wherein said rules include default conditions for determining which of said plurality of said media segments to use when no appropriate information regarding said target audience is available.
 6. The method of claim 1 wherein said media segments include audio, video, voice overs, and background music.
 7. The method of claim 1 wherein a subset of said plurality of media segments form a default generic message.
 8. The method of claim 1 wherein said assembly information includes data representing time segments; said media segments, and conditions.
 9. The method of claim 2 wherein said message for a specific targeted audience is assembled in a set top box for a television receiver contemporaneously with displaying said message to said specific targeted audience. 